An Exploration of the Impacts of Three Factors in Multimodal Biometric Score Fusion: Score Modality, Recognition Method, and Fusion Process

Operational applications for human identification require high credibility in order to determine or verify a person’s identity to a desired confidence level. Multimodal biometric score fusion (MBSF) can significantly improve detection, recognition, and identification performance of humans. The goals of this research are to explore the impact of each factor in a MBSF process and to determine the most important (key) factor. The following are three main factors that will be investigated and discussed in this paper: score modality, recognition method, and fusion process. Specifically, score modality is defined as imaging device (hardware) for biometric data acquisition. Recognition method is defined as matching algorithm (software) for biometric score calculation. A fusion process such as arithmetic fusion, classifier-based fusion, or density-based fusion, is used to combine biometric scores. The hidden Markov model (HMM) is also applied to the MBSF process as a baseline comparison. The accuracy of human identification is measured with a verification rate. A new metric, relative rate increase (RRI), is proposed to evaluate the performance improvement using score fusion. Several recognition methods (two to four matchers) and four fusion processes (mean, linear discriminant analysis, k-nearest neighbors, and HMM) are compared over four multimodal databases in our experiments. The experimental results show that the score modality is the dominant factor in biometric score fusion. The fusion process becomes more important in a single modality fusion. Adding more recognition methods into the fusion process has the least impact on fusion improvement.